Conversation
…amework Late Tasks can create new MonitorObjects based on existing MOs and QOs in the QC workflow. I called them "late", because they would be typically executed close to the end of the processing chain. When running QC workflows distributed over multiple nodes, with Mergers combining the results, they can be used to process the merged results. The adequate use cases involve: * creating trends and correlations from other MonitorObjects and QualityObjects (trending a histogram average, trending quality) * creating plots which can only be constructed from merged objects (ratios, visualizations, ...) * creating summary canvases of QualityObjects available in the QC workflow They are meant to slowly replace Post-Processing, at least in the use cases mentioned above.
305c575 to
1541053
Compare
knopers8
changed the title
Barthelemy
left a comment
Barthelemy
left a comment
There was a problem hiding this comment.
First of all, let me apologize for this very late review.
This is a great job! Thank you.
The late tasks have clearly the potential to replace the PostProcessing tasks in many cases.
I have put a few questions and comments that I let you address.
| @@ -37,9 +41,10 @@ Before developing a module, one should have a bare idea of what the QualityContr | |||
|
|
|||
|  | |||
There was a problem hiding this comment.
I have to update that and upload it after this is merged.
| } | ||
| } else if (matchingDataSource->isOneOf(DataSourceType::Check, DataSourceType::Aggregator)) { | ||
| auto qo = DataRefUtils::as<QualityObject>(ref); | ||
| auto key = qo->getName(); |
| auto mo = dynamic_cast<MonitorObject*>(obj); | ||
| if (mo != nullptr) { | ||
| if (matchingDataSource->subInputs.empty() || std::ranges::find(matchingDataSource->subInputs, mo->getName()) != matchingDataSource->subInputs.end()) { | ||
| taskInputs.insert(mo->getName(), std::shared_ptr<MonitorObject>(mo)); |
There was a problem hiding this comment.
I don't remember if we do that elsewhere, but using the MO name is a bit risky in case there are several tasks publishing an object with the same name (given that we have multiple sources). Or are we sure that we receive only data from one given task ?
| lts.moduleName = lateTaskTree.get<std::string>("moduleName"); | ||
| lts.detectorName = lateTaskTree.get<std::string>("detectorName", lts.detectorName); | ||
|
|
||
| lts.outputActivityStrategy = outputActivityStrategyFromString.at(lateTaskTree.get<std::string>("outputActivityStrategy", "integrated")); |
There was a problem hiding this comment.
I guess that we do it like that everywhere, but basically here we will just fail if we put "integration" instead of "integrated". Perhaps at some point we should better handle wrong keywords from the config.
| { | ||
| const QCInputs& taskInputs = extractInputs(pCtx); | ||
| if (taskInputs.size() == 0) { | ||
| ILOG(Warning, Support) << "Could not prepare inputs for task, will not call process() and publish objects. " |
There was a problem hiding this comment.
in which case can this happen ?
| { | ||
| using traits = ActorTraits<ConcreteActor>; | ||
| return dataProcessorName(detectorName, userCodeName, traits::sActorTypeKebabCase); | ||
| return dataProcessorName(userCodeName, detectorName, traits::sActorTypeKebabCase); |
There was a problem hiding this comment.
was there a bug here ?
There was a problem hiding this comment.
yes, the arguments were mistakenly swapped. i can make a separate PR maybe.
| CustomParameters customParameters; | ||
|
|
||
| // reco | ||
| // GRPGeomRequestSpec grpGeomRequestSpec; |
There was a problem hiding this comment.
more of a todo for later, but i'll create a ticket instead.
This branch has not been deployed
2 participants
Late Tasks can create new MonitorObjects based on existing MOs and QOs in the QC workflow.
I called them "late", because they would be typically executed close to the end of the processing chain.
When running QC workflows distributed over multiple nodes, with Mergers combining the results, they can be used to process the merged results.
The adequate use cases involve:
They are meant to slowly replace Post-Processing, at least in the use cases mentioned above.
Needs #2660 to have [WIP] removed.